1. Field of the Invention
The present invention relates to techniques for testing a software system, and particularly to a technique for narrowing down test cases when repeatedly executing tests.
2. Description of the Background Art
Conventionally, there have been various known software system development methodologies, including the “waterfall development methodology”, the “prototype development methodology” and the “spiral development methodology”. Software system development phases of these various development methodologies include “requirements definition”, “designing”, “programming”, “testing” and so on. Among these phases, testing of a software system is generally carried out in accordance with a test specification. The test specification describes for each test case a test method, conditions for determining a pass or fail (a success or failure), and so on.
In software system development, the above-mentioned phases are sometimes repeated. In such a case, testing is repeatedly performed on test cases originally generated during development or test cases generated in accordance with, for example, specification changes. However, in the software system development, it is sometimes difficult to test all test cases for reasons of limitations in development time, human resources and so on. In such a case, it is necessary to select test cases that are to be tested in the current phase from all test cases. Where the number of test cases is small, test case selection can be carried out relatively easily even if it is carried out manually. However, as the number of test cases increases, the manual test case selection becomes difficult.
Therefore, conventionally, test case selection in accordance with, for example, importance levels of test cases are carried out by computer software processing. Specifically, the test cases are classified into ranks H, M and L (High, Middle and Low, respectively) in accordance with their importance levels, and the test case selection is performed, for example, such that only rank-H test cases are tested or only rank-H and rank-M test cases are tested.
Japanese Laid-Open Patent Publication No. 2003-99283 discloses a design support method for designing optimal test cases, in which system characteristics are represented by characteristic values based on a plurality of evaluation perspectives, regarding specification items extracted from a system specification, and a testing strategy suitable for the system targeted for the testing is generated and selected, thereby deriving testing priority levels indicating specification items that are to be preferentially tested.
However, according to the above-described conventional art, the test cases are automatically selected in accordance with their priority or importance levels, which are set based on characteristics, etc., of the software system, and therefore it is possible to save a test case designer or administrator the trouble, but it is not possible to suitably change the set priority or importance levels, and thereby to change the number of test cases that are to be selected. In particular, it is necessary to repeatedly select suitable test cases (specifically, a suitable number of test cases) through a trial and error process, considering various factors, such as the set priority or importance levels, and thereby to predict (simulate) an achieved status of testing, in order to enhance the quality of testing, i.e., the quality of the software system that is to be tested, considering a limited remaining development time. However, the above-described conventional art does not particularly take such changes into consideration.
In addition, an apparatus user, such as the designer or administrator, might determine some test cases to be tested without fail, regardless of the set priority or importance levels, or conversely some test cases might be determined not to be tested. However, according to the above-described conventional art, it is not possible to select the test cases based on such determinations.